A Model-Based Namespace Metadata Benchmark for HDFS
نویسندگان
چکیده
Efficient namespace metadata management is increasingly important as next-generation storage systems are designed for peta and exascales. New schemes have been proposed; however, their evaluation has been insufficient due to a lack of an appropriate namespace metadata benchmark. We describe MimesisBench, a novel namespace metadata benchmark for next-generation storage systems, and demonstrate its usefulness through a study of the scalability and performance of the Hadoop Distributed File System (HDFS).
منابع مشابه
DNN: A Distributed NameNode Filesystem for Hadoop
The Hadoop Distributed File System (HDFS) is the distributed storage infrastructure for the Hadoop big-data analytics ecosystem. A single node, called the NameNode of HDFS stores the metadata of the entire file system and coordinates the file content placement and retrieval actions of the data storage subsystems, called DataNodes. However the single Na-meNode architecture has long been viewed a...
متن کاملHigh Scalability of HDFS using Distributed Namespace
In data intensive computing, Hadoop is widely used by organizations. The client applications of Hadoop require high availability and scalability of the system. Mostly, these applications are online and their data growth rate is unpredictable. The present Hadoop relies on secondary namenode for failover which slows down the performance of the system. Hadoop system’s scalability depends on the ve...
متن کاملScaling File System Metadata Performance With Stateless Caching and Bulk Insertion
The growing size of modern storage systems is expected to achieve and exceed billions of objects, making metadata scalability critical to overall performance. Many existing parallel and cluster file systems only focus on providing highly parallel access to file data, but lack a scalable metadata service. In this paper, we introduce a middleware design called IndexFS that adds support to existin...
متن کاملWhat Terms Does Your Metadata Use? Application Profiles as Machine-Understandable Narratives
The SCHEMAS Registry aims at providing a selected and annotated overview of metadata vocabularies and their use in application environments. Based on harvested metadata in RDF (Resource Description Framework), the registry allows users to explore links between "namespace schemas", which declare standard definitions of metadata terms, and "application profiles" – RDF statements about the use or ...
متن کاملImproving small file performance in object-based storage
This paper proposes architectural refinements, server-driven metadata prefetching and namespace flattening, for improving the efficiency of small file workloads in object-based storage systems. Server-driven metadata prefetching consists of having the metadata server provide information and capabilities for multiple objects, rather than just one, in response to each lookup. Doing so allows clie...
متن کامل